Optimal allocation of resource for surrogate modelling

We address the issue of fitting a surrogate to data generated from numerical simulators that have tunable fidelity. In particular, we focus on problems where data is generated using Monte-Carlo simulations. We show that this problem is similar to optimal design of experiments (DOE) but includes additional variables: the accuracy of the response at each point, and a computational time constraint. Since full optimization is often infeasible, we propose two alternatives to obtain nearly optimal designs: fixing a spatial DoE and optimizing only with respect to the fidelities, or constructing sequentially the design for both training point locations and fidelities. Finally, we apply the two techniques to the fitting of polynomial models, and show substantial improvement in accuracy using optimal allocation of resource.